This pull request adds comprehensive OpenTelemetry‑based tracing.
It adds a set of GraphQL‑specific spans, connects the Router to Hive Console's Tracing and other OpenTelemetry exporters (via OTLP).

Goals of this Pull Request:

• Introduce OpenTelemetry tracing
• Integrate with Hive Console's Tracing
• Support OTLP exporters (gRPC and HTTP)
• Tracing propagation (trace context, baggage, jeager, b3/zipkin)

Things out of scope for this Pull Request:

• No debug-level spans (we can add them when needed)
• Customization and filtering of spans (future Pull Request)
• Metrics (after the point above is done)
• Logs (after the work on polishing the logs is done)

Code changes

Almost all code that is responsible for tracing and setting up OpenTelemetry is in internal crate's telemetry directory.
You will see there span definitions, exporters setup, Hive Console Tracing integration and things like that.

Every span created by Hive Router has hive.kind attribute that describes what kind of action the span represents. This is useful to filter spans without relying on span names, which can change over time.

Performance costs?

To minimize the performance impact of tracing to minimum, spans are only created when tracing is enabled. All the computation that is needed to create attributes is also only done when tracing is enabled. I spent some time making sure the overhead is zero or near-zero when tracing is disabled.

With tracing enabled, the overhead is still as low as I could make it, by avoiding allocations, using static strings for attributes names and avoiding complex computations when creating spans and attributes. There's of course some cost (TODO: measure it once again and get the numbers).

How spans are created and used?

Spans are created using tracing::span_info. Every "phase" has its own span and a struct that helps with creating and managing it. This is intentional, to store all span-related logic in one place and make it easy to understand what attributes are set on each span, and what methods are available to manipulate the span and not pollute the rest of the codebase with span-related logic. Imagine looking for the parsing span and the attributes if it's not centralized in one place.

For example, the parsing phase has GraphQLParseSpan struct that creates a span and has the record_cache_hit method to record whether the parsing was a cache hit or miss.

The tracing crate "requires" attributes to be static strings, and predefined at the span creation. This is for performance reasons.
That's why all the standard attributes are there from the beginning, that we later on fill in with data, like previously mentioned record_cache_hit method that fills in the cache.hit attribute.

You will notice that attributes names are defined as constants in the telemetry::attributes module, but not used when the spans are created. The reason for it is that tracing crate requires attributes to be static strings, so we can't use variables there.
I stored them as constants to avoid typos and have a single source of truth for attribute names, when we need to refer to them later on (like when to fill them in with data, or transforming them for many reasons). This gives us some kind of type safety.
To fillup the gap between static strings used for span creation and constants defined in the attributes module, I added a bunch of tests that verify that all attributes defined in the attributes module are actually used when creating spans. This way we can be sure that there are no typos and that all attributes are defined and match.

Spans are started and stopped automatically by either using the guard pattern (calling span.enter() returns a guard that stops the span when dropped) or instrumenting the futures with the span (using instrument method).
Spans are created by calling ::new() method on the span struct, which returns the struct instance. The span itself is stored inside the struct.

What exporters are supported?

Currently, two exporters are supported:

• Hive Console Tracing exporter
• OTLP exporter (gRPC and HTTP)

The OTLP exporter can be configured to send traces to any OTLP-compatible backend, like Jaeger, Zipkin, Honeycomb, NewRelic, Datadog and many others.

I tried to make everything configurable, so you can enable/disable specific exporters, set endpoints, headers and other options via environment variables or expressions.
Even the collection of spans can be configured, the batching options can be set, so users can tune the performance of the tracing to their needs.

Semantic conventions for HTTP

There's a standard for HTTP semantic conventions defined by OpenTelemetry for attributes, but not every tool supports it yet.
In Hive Router, we support both deprecated attributes and the stable ones, so that we can interoperate with more tools.

This is also configurable, so users can choose which set of attributes they want to use. We have three modes:

• spec_compliant: use only stable attributes
• deprecated: use only deprecated attributes
• spec_and_deprecated: use both sets of attributes

How deduplicated requests are traced?

The http.inflight span is created for each outgoing HTTP request to subgraphs when request deduplication is enabled. Multiple concurrent identical requests will create separate http.inflight spans in different traces, but only the leader executes the HTTP call.

Leader role (hive.inflight.role="leader"):

• First request to win the deduplication race
• Creates http.client as a child span to execute the actual HTTP request
• All joiners wait for this leader's result

Joiner role (hive.inflight.role="joiner"):

• Subsequent identical concurrent requests
• Does not create http.client (that span exists in the leader's trace)
• Waits for the leader's HTTP response
• References leader via hive.inflight.key (same fingerprint value)

Cross-trace correlation: All inflight spans (leader + joiners) for the same deduplicated request share the same hive.inflight.key value, allowing observability tools to correlate them across different traces. In future we may introduce a span link.

Leader trace (first/winning request):

GraphQL Subgraph Operation
  └─ http.inflight (role="leader", key=12345)
      └─ http.client  <- executes the actual HTTP request

Joiner trace (deduplicated request):

GraphQL Subgraph Operation
  └─ http.inflight (role="joiner", key=12345)
      └─ [no http.client - it's in the leader's trace]

Changes in Hive SDK

Hive Console's Tracing requires the operation hash to be sent as an attribute, and represent the same hash as in Hive SDK (Usage Reporting). I made changes to Hive SDK and exposed a function to compute the operation hash, and used it in Hive Router to set the hive.operation.hash attribute on the graphql.operation span.

I noticed that Hive SDK relies on graphql_parser to minifiy and normalize the query before hashing it, but it's super inefficient (we do Document to String conversions etc) that I hope to reimplement in a more efficient way in near future, but that requires forking and owning the graphql_parser crate.